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ABSTRACT 

Naval Aviation’s annual Class ‘A’ Flight Mishap rate is commonly used as a 
measure of safety effectiveness. Interventions implemented over the past four decades 
greatly reduced mishap occurrence by focusing on aircrew and supervisory error. Less 
attention has been paid to the role maintenance plays in Naval Aviation mishaps, though 
it is consistently responsible for approximately 16 percent of all Class ‘A’ Flight 
Mishaps. In 1998, a Maintenance Climate Assessment Survey (MCAS) was developed to 
evaluate safety concerns from the perspective of an aircraft maintainer. This thesis 
utilized the revised MCAS to assess its validity and utility as a diagnostic tool to access 
several aircraft communities within the Naval Reserve. It proved useful in aiding 
Commanders and Aviation Safety Officers (ASOs) in evaluating their maintenance 
operation’s safety posture. The results of this study produced a finalized MCAS for fleet- 
wide distribution. The findings will serve to encourage proactivity within aviation 
maintenance in the areas of safety awareness and risk management. This tool will also 


aid the monitoring of ongoing safety programs or implementation of new ones. 
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EXECUTIVE SUMMARY 

Naval Aviation has done a remarkable job cutting its mishap rate in half each 
decade between the 1950s and 1990s. Vast improvements in aircraft technology, coupled 
with a high degree of standardization have reduced the role that mechanical failures play 
in aircraft mishaps to an all time low. Standardization and aggressive training has also 
reduced the role that human error plays in mishaps. Despite the overall decrease, human 
error related causal factors seem to have leveled off, and in recent years are even 
increasing. In recent years, a Human Factors Quality Management Board (HFQMB) has 
had much success by focusing on evaluating the contribution aircrew error plays in Class 
‘A’ mishaps. 

The initial progress achieved by the HFQMB was impressive. In 1997, Naval 
Aviation experienced its second lowest Class *A’ mishap rate of all time, with the Navy 
recording its all time low. It achieved this by applying the practices and procedures 
obtained through the study of high-risk organizations (HROs); organizations that though 
operating in high-reliability environments sustained high rates of safety. This success 
prompted a look into reducing maintenance-related mishaps (MRMs) via the same 
approach of mishap data analysis, benchmarking, and climate safety assessment. 

This thesis is an extension of the efforts to reduce MRMs by assessing the 
prevailing attitude toward safety held by those who actually perform aircraft 
maintenance. It entails the analysis of data obtained through the administration of a 
survey containing questions based on an existing Model of Organizational Safety 
Effectiveness (MOSE). The results of this thesis are intended to provide a better 


understanding into the possible human factors involvement in MRMs and furnish the 


xV 


Squadron Commander and Aviation Safety Officer (ASO) with a tool for assessing the 
safety posture of their unit’s maintenance operations. 

The study encompassed 439 maintenance personnel across three different aircraft 
communities within a common Naval Air Reserve Force (NARF). The questionnaire 
consisted of 35 items, each rated according to a five-point Likert scale (strongly disagree, 
disagree, neutral, agree, strongly agree). The questions onginated from each of the six 
MOSE categories: Process Auditing, Reward System, Quality Assurance. Risk 
Management, Command and Control, and Communication and Functional Relationships. 

Basic data exploration and descriptive Statistics were performed on the data set to 
determine its underlying distribution and summarize respondents’ answers. Principal 
component analysis and clustering analysis were conducted to determine if the survey 
was dominated by any particular MOSE component(s). Analysis of vanance (ANOVA), 
and multiple comparison testing were performed on the data to identify any 
commonality/differentiation among the various communities with respect to their 
prevailing attitude toward aviation safety in maintenance operations. 

The results of this study show that a squadron’s safety posture can be accurately 
assessed through the use of the MCAS. The analysis identifies no significant difference 
between each of the three communities surveyed with respect to their overall attitude 
toward safety during the conduct of aviation maintenance. Each community displays a 
positive outlook with respect to the way day-to-day maintenance operations are 
conducted. 

Analysis also reveals that despite the overall favorable attitude, some areas of 


concem do exist. These concems target perceptions of inappropniate staffing levels and 
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communication breakdowns that occur due to the overcommilting of limited personnel. 
These concerns can serve as Starting points for the implementation of intervention 
Strategies. By linking problem areas to their corresponding MOSE components, these 
interventions can be specifically tailored to achieve maximum applicability. This study 
concludes with the restructuring of the MCAS, which results in an increase from 35 to 40 
questions. It is believed that the increase in survey size is outweighed by the benefit of 
Clarity obtained by the restructuring. 

By utilizing the MCAS, evaluation of a squadron’s safety posture can be assessed 
on a continual basis. Problem areas can be identified and cross-referenced to specific 
MOSE components, which can aid Naval Aviation in the implementation of procedures 
that directly target threats to safety. An organization need not wait for a mishap to occur 
in Order to make changes in its safety posture. For it is only through a proactive stance 
that Naval Aviation can achieve for a lower mishap rate in today’s high paced operating 


environment. 
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I. INTRODUCTION 

Naval Aviation’s current operational tempo, manning shortages, and lengthier 
deployments pose significant challenges to maintaining an aging fleet of aircraft. Present 
(and forecasted) declining retention rates will exacerbate this situation, impairing mission 
readiness and potentially fostering mishaps (Dake, 1998; Mann, 1998). Further, 
increased pressure to meet mission requirements can prompt personnel to compromise 
safety by taking shortcuts or skipping procedures, often resulting in disastrous situations 
(Perrow, 1984). The combination of these factors serve to alter an organization’s culture 
and its perspective on safety posture (Pidgeon, 1991). This thesis focuses on the safety 
posture across differing aircraft communities within a common Naval Reserve Wing. It 
will attempt to determine if the aforementioned conditions pose a threat to the safety 
climate among the communities’ maintenance operations. This study also parallels 
similar research being conducted by Goodrum (1999), that concentrates on the prevailing 


safety attitude among differing squadrons within a common community. 


A. BACKGROUND 

Naval Aviation has done a remarkable job cutting its mishap rate in half each 
decade between the 1950s and 1990s as depicted in Figure 1. Vast improvements in 
aircraft technology, coupled with a high degree of standardization has reduced the role 
that mechanical failures play in aircraft mishaps to an all time low as depicted in Figure 
2. Standardization and aggressive training has also reduced the role that human error 
plays in mishaps. Figure 2 also reveals that despite the overall decrease, human error 


related causal factors seem to have leveled off, and in recent years are even increasing. 


This relationship 1s possibly due to the current pressures imposed by a high state of 
operational tempo coupled with personnel downsizing and increased aircraft technical 
complexity (Rhame, 1999). This marked trend, and a sequence of events in 1996 caused 


Naval Aviation leadership to reevaluate the way it conducts business. 


776 aircraft 
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Figure 1. FY 90-96 Naval Aviation Flight Mishap Rates and Intervention Strategies. 
(Naval Safety Center, HFACS Brief, 1997). 
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Figure 2. Mechanical Failures vs. Human Errors in Naval Aviation Mishaps. 


(Naval Safety Center, HFACS Brief, 1997). 


In 1996, a string of 17 Class ‘A’ mishaps occurred within a 75 day period 
culminating with a Navy F-14 “Tomcat” crashing into a residential Nashville, TN 
neighborhood killing not only the aircrew, but some local residents as well (Nutwell & 
Sherman, 1997). Class ‘A’ mishaps are defined as Naval aircraft incidents resulting in 
death, permanent disability, or loss of over one million dollars (OPNAV 3750.6Q, 1989). 
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Presently, it’s accepted that at least 80 percent of all Class ‘A’ mishaps since 1990 list 
human error as a contributing causal factor. As a result, Vice Admiral Brent Bennett, 
then Commander Naval Air Forces Pacific, established the Human Factors Quality 
Management Board (HFQMB). The HFQMB’s purpose was to analyze and improve the 


processes, programs and systems that impact human performance in Naval Aviation with 


a goal to cut the current mishap rate in half by the year 2000 (Nutwell & Sherman, 1997). 


The HFQMB (Nutwell & Sherman, 1997) concentrates on three main areas: (1) 
Class ‘A’ Flight Mishaps, (2) Organizational Benchmarking, and (3) Command Safety 
Assessment (CSA). The objective of evaluating Naval Aviation Class ‘A’ Flight 
Mishaps is to analyze past human factor involvement. As depicted in Figure 3, this 
evaluation identifies supervisory and aircrew factors to be major contributors in Class ‘A’ 
Flight Mishaps. The HFQMB elected at this point to concentrate on these two factors 
during its early phases. The benchmarking effort examines programs influencing aircrew 
performance. Among the successful strategies examined in similar organizations were 
Operational Risk Management (U.S. Army) and Line Oriented Flight Training (United 
Airlines). Many of these programs were selected for immediate consideration by the 


HFQMB for adoption and implementation given tailoring, resources and support. 
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Figure 3. FY 90-96 Naval Aviation Class A Flight Mishap Causal Factors. 
(Naval Safety Center, HFACS Brief, 1997). 


The CSA survey as a climate evaluation instrument determines a command’s 
safety posture from an aircrew perspective, which was based on the level of involvement 
of aircrew caused mishaps (Civarelli & Figlock, 1996). The CSA survey is based on a 
model of High Reliability Organizations (HROs) and consists of five basic components: 
Process Auditing (PA), Reward System (RS), Quality Assurance (QA), Risk 
Management (RM) and Command i Control (CC). The CSA, in a survey of 67 
squadrons, effectively uncover deficiencies and strengths in Naval Aviation (Nutwell & 
Sherman, 1997). For example, it showes significant room for improvement in the areas 
of risk management, human factors evaluation, and aircrew coordination training. This 
Survey is now a tool being provided to all Commanding Officers and Aviation Safety 
Officers (ASOs) for self-assessment of their command’s flight operations safety posture 
(R. Figlock, personal communication, 1999). 

The approach taken by the HFQMB has shown merit; in 1997 the U.S. Navy 
experiences its lowest Class ‘A’ Flight Mishap rate, and Naval Aviation as a whole has 
its second lowest rate. In 1998, the U.S. Marine Corps has its lowest Class ‘A’ Flight 
Mishap rate as a service. Despite this success, the HFQMB is still far from achieving its 
Stated goal of a 50 percent reduction in human error mishaps. This necessitates a shift to 
impact areas in addition to supervisory and aircrew error. The HFQMB forms a Human 
Factors Maintenance Process Action Team (PAT). The PAT develops a charter similar to 
that of the HFQMB while developing its three pronged approach of Mishap Data 
Analysis, Benchmarking, and Command Safety Assessment. 

To date, there has been markedly less interest concerning the impact of human 


factors on military aviation maintenance operations. The primary reason being that 


maintenance related mishaps (MRMs) generally account for only 15-17 percent of the 
Class ‘A’ Flight Mishaps. However, though maintenance is involved in a small number 
of Class A’s, it is involved in a relatively large number of Class C’s. Very little has been 
published about human factor involvement in Naval Aviation maintenance at either the 
organizational or individual worker level. Indeed, a search of published references from 
1976-1988 resulted in only 15 papers on the subject. This lack of interest was also shared 
by the commercial airline industry (Anonymous, 1999). 

In the 1990s the Federal Aviation Administration (FAA) initiated a large-scale 
campaign to combat human factors in maintenance and inspection (FAA website). Their 
efforts to date have focused on the identification of maintenance error factors and 
Strategies for their reduction, similar to the mishap data analysis and benchmarking thrust 
of the HFQMB. The only area that was not touched upon at that time was the safety 
climate and its assessment. 

Noting this void, Baker (1998), in conjunction with Naval Postgraduate School 
(NPS) staff and Commander Naval Air Forces Pacific (CNAP) personnel, develops a 
maintenance version of the CSA that assesses the human factor involvement in aircraft 
maintenance from the perspective of the maintainers themselves. The Maintenance 
Climate Assessment Survey (MCAS) is administered to various Navy Reserve squadrons 
in California. Baker’s goal is to gain a stronger understanding of the HRO factors within 
the reserve Naval Aviation maintenance community that contributed to safety / risk 


management. 


Be PURPOSE 

The purpose of this study is to administer and evaluate a questionnaire survey in 
an attempt to gain a broader understanding of the HRO factors within Naval Aviation 
across different aircraft communities. This study utilizes a revised survey developed by 
Baker (1998) as part of a HFQMB sponsored effort. This 35-item survey (reduced from 
an original prototype of 67 items) is the end result of Baker’s NPS Master’s Thesis 
Research. Results from this survey will be utilized to improve our understanding of 
possible influences that human factors play in aviation maintenance operations. 
Additionally, the results of the survey will provide the basis for assessing a Squadron’s 
posture toward safe maintenance practices, with the intent of identifying potential areas 


for intervention. 


C PROBLEM STATEMENT 

Human error in aviation maintenance, however small a role it plays in Class ‘A’ 
Flight Mishaps, requires that its contribution in the causal chain of events be understood. 
Through administration of the MCAS, an attempt will be made to gain insight to a 
Squadron’s overall posture toward safety and safe maintenance practices in accordance 
with the model developed out of the evaluation of high-risk organizations. This thesis 
will explore the following questions: 


ih. Does the MCAS capture an overall attitude toward aviation maintenance 
safety from the maintainer’s perspective? 


om Are there any discernable differences between squadrons / communities 
surveyed and / or does there appear to be any consensus? 


ay Can the MCAS be refined further to either reduce its size or clarify any 
questions? 


D. SCOPE AND LIMITATIONS 

The intent of this study is to gain a better understanding of the maintainer’s 
perspective on safety within Naval Aviation maintenance organizations. Select 
communities of the Naval Air Reserve Force (NARF) are surveyed due to the interest of 
the Commander, NARF, and the desire to assess the safety attitude across homogeneous 
populations. The only responses used in this analysis are those of Naval Aviation 
maintenance personnel. Incomplete surveys are omitted. 

The next chapter will provide a literature review of the history of safety practices 
in civil and military aviation. Additionally, the concept of high-reliability organizations 
and the safety concepts employed by them will also be discussed. Methodology 1s 
discussed in Chapter III. Results of the study will be covered in Chapter IV, followed by 
conclusions and recommendations in Chapter V. Appendices provide supplemental 


information to augment material found throughout the body of the thesis. 


Il. LITERATURE REVIEW 


An in depth hterature review 1s conducted during the compilation of this thesis. It 
will begin with an overview of the importance of safety among aviation and high- 
reliability organizations. Next, a history of safety awareness in both civilian and military 
aviation is covered. The chapter finishes by discussing high-reliability organizations and 


the concept of corporate safety culture. 


A. OVERVIEW 

Naval Aviation is by its very nature a dynamic organization. But the nature of its 
being instills in it an inherent amount of danger. Indeed, the combined costs of all United 
States Navy (USN) and United States Marine Corps (USMC) mishaps was over four 
billion-dollars for fiscal years (FY) 1993-1998. Of this, aviation mishaps account for 3.8 
billion dollars (95 percent) of the total cost. For Fiscal Year (FY) 1998, aviation mishaps 
account for 89 percent of total USN mishap costs and 93 percent of all USMC mishap 
costs. (Naval Safety Center, 1998). 

Perrow (1984) states that high-risk organizations (HROs) are those which embody 
Special characteristics beyond their normal dangers that make accidents in them 
inevitable, even normal; e g., nuclear power plants and petro-chemical plants. Naval 
Aviation (as with aviation in general) is considered to be a HRO (Figlock, 1998). Often 
these organizations are heavily regulated, and are required by law to adhere to strict 
safety guidelines (Eiff & Mattson, 1998). This in part accounts for the reason these 
HROs are able to operate in a relatively safe manner. However, there 1s another factor 
that impacts the success of these organizations; and that is the degree to which each HRO 


incorporates a cultural attitude toward safety. The term culture is defined as a collection 


of norms, beliefs, roles, attitudes, and practices of a given group or organization (Tumer, 
1991). Naval Aviation can strive to achieve higher levels of safety than it currently 
enjoys by studying these “culturally safe” organizations and implementing their methods. 

Safety is important to Naval Aviation for mishaps are costly and have an adverse 
effect on combat readiness. In addition, the Naval Safety Center (NSC) estimates that 
over the past decade the average cost per Class A mishap has quadrupled from $5.1 
million to over $20 million per mishap (NSC, 1997). In addition, the USN and USMC 
accept delivery of only 40 new aircraft in FY-97 while losing 24 to mishaps (NSC, 1998). 
Since a number of aircraft are struck from the inventory each year due to age, we are hard 
pressed to keep par with aircraft requirements. With Department of Defense budgetary 
constraints/cutbacks that no longer permit the military to purchase as many aircraft as it 
once did, lost aircraft pose a void that cannot be easily filled (Figlock, 1998). 

Personnel are also affected by mishaps. The most damaging effect of a mishap on 
personnel is the loss or impairment to human life. Whether permanent or temporary, this 
mishap result has a psychological effect that serves to further reduce a unit’s effectiveness 
and morale. In addition, mishaps impact productivity. Productivity is reduced from either 
the temporary or permanent loss of personnel directly involved with the mishap. Other 
personnel detracted from their primary billets, due to their participation in either the mishap 
investigation or reclamation process further reduces a unit’s ability to carry out its mission. 
With personnel shortages effecting all commands throughout the military, mishaps only serve 
to exacerbate this situation. In the end, costs of aviation mishaps (both to aircraft and 
personnel) pose a threat to Naval Aviation’s ultimate goal, which is to maintain a high level 


of combat readiness. (Cressy, 1998: Mann, 1998). 


10 


B. SAFETY AWARENESS IN AVIATION 


1. Historical Roots 


Increased safety has long been an issue of concern to those involved with 
aviation, dating as far back as 1908 with the first fatal air crash. Founded in 1918, the 
United States Airmail Service stages its first strike in 1919 over safety concerns (Perrow, 
1984). The pressures on pilots to fly to meet government and business mail schedules, 
coupled with aircraft reliability of the times result in a life expectancy of four years. 
Thirty-one of the first forty pilots die under these conditions (Perrow, 1984). The Army 
Air Corps’ attempts at mail delivery prove equally as disastrous. Beginning service on 
February 16, 1934, they lose six pilots in the first week. During the next month, six more 
airman are killed (four of them in a single day) before systems and procedures are 
established to increase mission safety (Heppenheimer, 1995). 

Despite the hazards associated with this newfound occupation, the opportunities 
the airmail service present are apparent and measures are taken to improve its overall 
operation and safety. Around 1920, under the direction of E.H. O’Shaughnessy, the 
airmail service establishes new standards for all applying pilots. These require that all 
applicants have a minimum of 500 flight hours, pass a qualifying examination and a 
medical examination prior to being hired. Additionally, aircraft receive frequent 
inspections, with airframes and engines being overhauled according to a periodic 
maintenance schedule. (Heppenheimer, 1995). 

The idea of commercial aviation, an industry where people paid to fly in aircraft, 
begins in 1910 with the Delag Airship Company’s use of Zeppelins in Germany 
(Academic American Encyclopedia, 1994). Starting that year, experiments are conducted 


in the use of airplanes to carry mail, freight, and later passengers (Chambers’s 


1} 


Encyclopedia, 1973). In 1914, the St. Petersburg-Tampa Airboat line operates across 
Tampa Bay in Florida; and in Russia, a multiengine transport plane flies from St. 
Petersburg to Kiev (Chambers’s Encyclopedia, 1973). Despite these early strides, it is 
not until after the First World War that commercial aviation truly sets its start. Surpluses 
of military aircraft, especially bombers, are easily converted to carry passengers. Many 
European countries take an early lead in the advancement of commercial aviation. Their 
railroad systems, having suffered extensive damage from the war, provide an opportunity 
and impetus for early airline entrepreneurs. By the end of 1919, Europe boasts quite an 
extensive network that links many of its more important cities, including a daily 
international service from London to Paris (Heppenheimer, 1995). 

The United States gets off to a slow start with respect to commercial aviation. 
Indeed, at the time of Charles Lindbergh’s solo Trans-Atlantic flight in 1927, the U.S. has 
only thirty planes that can be considered airliners, offering no more than two hundred 
seats (Heppenheimer, 1995). But Lindbergh’s feat spurs such an interest 1n aviation that 
the number of tickets purchased for commercial flights in 1926 (5,800) increases to 
417,000 by 1930 (Heppenheimer, 1995). The U.S. soon establishes itself as the leader in 
commercial aviation and has maintained that title to this day. Accomplishments of the 
Airmail service and technological improvements are used to advance the industry. With 
the application of jet propulsion to commercial air transportation in 1952 (by British 
Overseas Airline Corporation), more efficient lift capability and faster travel times lead to 
the introduction of low-fare air tourism to society (Heppenheimer, 1995). This will 
eventually evolve the industry into one that today carries over 800 million people 


annually (American Academic Encyclopedia, 1994). 


A strong partnership between government and private industries is required for 
the initial development of aviation. Governmental regulation is crucial to the initial 
growth of commercial aviation and continues to play a role until the Air Transport 
Deregulation Act of 1978 (Academic American Encyclopedia, 1994). In Europe before 
World War I, scientifically trained engineers form research organizations and 
laboratories, the most notable of these being Britain’s Advisory Committee for 
Aeronautics (1909) and Ludwig Prandtl’s aerodynamic research center at GOttingen, 
Germany. In the United States, the National Advisory Committee for Aeronautics, the 
predecessor to the National Aviation and Space Administration (NASA), is established in 
1915. (American Academic Encyclopedia, 1994). 


Je Contemporary Thought 


As aviation becomes more commercialized, it inherently becomes more 
concerned with the issues of safety (Perrow, 1984). Public sentiment demands a 
continuing, affordable and safe air transportation system. Meshkati (1997) states that 
“Safety is an emerging force in business with dual financial and marketing effects.” 
Airline travel decreases after large accidents; airframe companies suffer if one of their 
models appears to have more than its share of problems (Perrow, 1984). Maintenance 
deficiencies that impact safety of flight can also be linked to flight delays, ground 
damage and other factors that have a direct impact on airline costs and business viability 
(Endsley & Robertson, 1999). Over the years, commercial aviation safety will appear to 


be molded by three various factors; oversight, technology and management. 


a. Oversight 


Oversight has taken the form of both regulation and research. The first 
federal attempt to impose safety regulations on civil aviation is made by the Air 
Commerce Act of 1926 (FAA Website). This created the Aeronautics Branch of the 
Department of Commerce, which assumes primary responsibility for aviation oversight. 
This requires the licensing of pilots and certifying of airplanes (amongst other duties). 

Since then, several legislative acts enlarge the scope of federal regulation 
concerning both air safety and commercial routes and fares. Such legislation eventually 
evolves the Aeronautics Branch into the Federal Aviation Agency in 1958 and to its 
present day form as the Federal Aviation Administration (FAA) in 1967. Under the 
Department of Transportation, the FAA regulates all aspects of aircraft safety and is 
instrumental in the implementation of standardization. (FAA Website, 1999). 

In addition to regulatory actions, combinations of incidents and accidents 
demand that certain attention be given to researching the causes of aviation mishaps or 
ways tO Improve maintenance safety (Rogers, 1991). The Socio-Technical Systems 
(STS) model, an organizational model used to understand purposeful work systems in 
complex environments for over 40 years, is been used to assess aviation heavy 
maintenance systems (Pidgeon & O’Leary, 1994). The Aviation Safety Research Act of 
1988 mandates that research attention be devoted to a variety of human performance 
issues including “aircraft maintenance and inspection”? (Human Factors in Maintenance 
Handbook, 1999). 


b. Technology 


Increases in technology greatly improve the safety of commercial aviation. 


In the 1930s, gains in wind tunnel testing, maintenance equipment and engine and 


airframe design combined to provide faster, larger and more durable airplanes (New 
Columbia Encyclopedia, 1975). The 1940s see the interruption of commercial aviation 
by the Second World War. But with its cessation comes the age of the jet engine and a 
coiresponding major change in aviation development that is further advanced by the use 
of jet aircraft in the Korean Conflict in the 1950s (Heppenheim, 1995). Remarkable 
advances in the electronics technology from the 1960s to today provide aircraft with 
State-of-the-art instrumentation, navigation and automation systems. Traffic Collision 
Avoidance System (TCAS), Global Positioning System (GPS), Ground Proximity 
Warning System (GPWS) and Instrument Landing System (ILS) are just a few examples 
of systems that technology has provided which enable aircraft to operate with confidence 
and safety in virtually any environment (Wiener & Nagel, 1988). 


c. Management 


Management can be used to manipulate the maintenance effort in order to 
achieve a safer working environment. Additionally, the management itself can be 
manipulated into bringing about changes that impact maintenance safety. In the area of 
maintenance management, “team-approaches” to conducting aircraft maintenance are 
examined. In 1985, Japan Air Lines (JAL) implements this theory by creating dedicated 
engineer/maintenance teams called “Kizuki” (meaning “airplane crazy”) which are 
responsible for a specific Boeing 747 aircraft at all times, regardless of its location 
(Human Factors in Maintenance Handbook, 1999). Also, the U.S. Air Force reports 
success in utilizing team-based aircraft maintenance organizations (Rogers, 1991). With 
respect to manipulating management itself, the Airline Pilot Association (ALPA) is a 


strong union among commercial airline pilots that is very concerned with aviation safety. 


It conducts its own studies and makes recommendations in the area of maintenance safety 
(Perrow, 1984). 

3. Military Aviation 

In the military, various intervention strategies are implemented over the years to 
increase safety and reduce the risks of mishaps. In Naval Aviation for example, the earliest 
of these dates back to 1912 with the Surgeon General’s requirement that all aviators pass a 
flight physical. This was followed shortly after with the institution of psychiatric 
evaluations following World War I (Bachman, 1918). More recent efforts take the form of 
engineering constraints: the introduction of the angled flight deck to U.S. aircraft carriers 
(1952); and systems/administration constraints: the creation of the Naval Safety Center 
(1954) and Fleet Replacement Squadrons (FRS, 1959); and the Naval Aviation Maintenance 
Procedures (NAMP, 1959), Naval Air Training and Operating Procedures Standardization 
(NATOPS, 1961) and the Squadron Safety Program in 1977 (Naval Safety Center Brief 
1997). By way of these programs, Naval aviation has effectively increased combat 
readiness by reducing aviation mishap rates (see Figure 1). 

With the 1990s, Naval Aviation (through the HFQMB) sees the implementation 
of a variety of human factor programs. These programs strive to modify the safety 
culture of communities by systematic and continual monitoring of the risk factors 
involved in military flight operations. The first, Operational Risk Management (ORM), 
is a decision making tool that systematically identifies, assesses and controls the risks 
involved with conducting a individual mission. The Aircraft and Aircrew Systems plan 
calls for accelerated installation of aircraft safety systems, such as flight data/cockpit 
voice recorders, ground proximity warning systems and midair collision avoidance 


systems. The Aircrew Coordination Training (ACT) program improves coordination and 


teamwork among the members of a given aircrew by way of computer and video-aided 
debriefs following aircrew simulation training (Nutwell & Sherman, 1997). Finally, the 
CSA (being based on the Model of Safety Effectiveness components) draws its strength 
from incorporating many of these proven high-risk traits. It’s utility as a tool for 
Commanding Officers and Aviation Safety Officers (ASOs) to assess their command’s 
safety posture 1s receiving increasing attention. 

These aforementioned programs are currently serving to facilitate and enhance the 
success of Naval Aviation’s current safety environment as evidenced by the 1997 Class 
‘A’ Flight Mishap rate of 1.96 accidents per 100,000 flight hours flown being the second 
lowest in history. To supplement these programs and encourage the development of 
new/additional ones, it is beneficial have an understanding of the complexities of high- 
risk organizations and how they can benefit from the concept of a corporate safety 


culture. 


C HIGH-RELIABILITY ORGANIZATIONS 

The notion of a poor safety culture receives widespread attention in accounts of 
the human and organizational errors underlying the 1986 Chernobyl nuclear power plant 
accident in the former Soviet Union (Turner, Pidgeon, Blockley & Toft, 1989; Pidgeon, 
1991). Analysis of the disaster prompts hindsight claims that Soviet engineers had a 
worse Safety culture than is present in the west. But despite this claim of a better safety 
climate, western industry sees its share of large-scale incidents. These include the 1979 
Three Mile Island nuclear power plant accident, the 1984 Union Carbide petrochemical 


plant gas leak, the 1986 Challenger space shuttle accident and the 1989 Exxon Valdez oil 


tanker spill. Although these accidents occur throughout a variety of organizations, they 
all have one thing in common- they are all high-risk organizations. That 1s, they are all 
organizations with catastrophic potential— the ability to take the lives of hundreds of 
people in one blow, or to shorten or cripple the lives of thousands or millions more 
(Perrow, 1984). 

What makes these HROs so potentially dangerous is that their level of technical 
complexity makes them vulnerable to accidents or failures that can not be predicted 
during the design process (Roberts, 1990). Though rare, catastrophic accidents tend to 
occur due to a myriad of interactions within components of the overall system. This 
“interactive complexity” is a characteristic of the system, not of a specific part or 
Operator, and is exaggerated when the system is “tightly coupled” (Perrow, 1984). 
Perrow describes systems as being tightly coupled when processes happen too quickly to 
be prevented. Usually, one malfunction triggers a malfunction within another 
independent part of the system and recovery from the initial disturbance is not possible. 
This is in contrast to “loosely coupled” systems, which Perrow describes as systems that 
can still have multiple failures, but these failures occur independently without one having 
caused the other. 

Organizations utilizing tightly coupled systems pose a much higher challenge to 
safety issues because they are difficult for an operator to diagnose and correct. 
Additionally, operator actions or the safety systems in place may only serve to exaggerate 
a malfunction while attempting to apply corrective action, thereby making things worse. 
This 1s because it may be some time before the nature of the problem is known. Risk will 


never be eliminated from these systems, but through experience, better designs and 


procedures, tight coupling can be reduced (Perrow, 1984). And it is through the 
implementation of a safety culture that these organizations can further reduce hazardous 


situations. 


D. CORPORATE SAFETY CULTURE 

A corporate safety culture is a specific set of norms, beliefs, roles, attitudes and 
practices within an organization which is dedicated to minimizing exposure of 
employees, managers, customers, ee and members of the general public to 
conditions considered to be dangerous or injurious (Turner, 1991; Pidgeon, 1991). The 
ultimate goal of a corporate safety culture is to minimize accidents by establishing a firm 
foundation in which safety is considered to be a priority. Characteristics of a good safety 
culture can be grouped into three areas— norms and rules for handling hazards, attitudes 
toward safety and reflection on safety practices (Pidgeon, 1991). 


1. Norms and Rules 


Norms and rules governing safety within an organization are at the heart of a 
safety culture. As guidelines for action, these shape the perceptions and actions of 
individuals within an organization as to what is and what is not to be regarded as a 
significant risk. Understanding this aids the individual in selecting an appropriate 
response when confronted with an unsafe situation (Turner, et al, 1989). A caution here 
is to guard against implying that the procedures established are the only rules by which 
all foreseeable hazards can be avoided. This leads to the inflexible or ritual application of 
existing rules to situations in which they may not apply. This is referred to as a form of 
cognitive “mind-set” or “groupthink” (Weick & Roberts, 1993). Preventing this pitfall 


requires a willingness to monitor ongoing technologies in diverse ways. This may 


include soliciting opinions about risk from outsiders; implementing positive reward 
structures for personnel who identify and report unsafe conditions, and encouraging 
creativity and imagination in developing and implementing safety procedures (Pidgeon, 
Ooh, 


jay Safety Attitude 


“Safety attitude refers to individual and collective beliefs about hazards and the 
importance of safety, together with the motivation to act on those beliefs” (Pidgeon, 
1991). This attitude can be achieved by the positive promotion of its corresponding 
ideals. But corporate cultures are notoriously resistant to change (Turner, et al, 1989). 
Change requires total dedication on the part of the organization, and anything less than 
total dedication will impact workers’ perceptions of management’s dedication to the 
seriousness of this topic. Zohar (1980) found that companies that have successful safety 
programs (and corresponding low accident rates) are those in which management 
maintains a strong commitment to safety. This commitment is demonstrated by active 
participation in safety activities on a routine basis. Zohar (1980) also found that 
companies with low accident rates are those in which safety officers hold a position of 
high rank or status. 

If approached correctly, the safety culture concept can create a synergetic affect. 
Tumer (1991) states that “one of the most potent ways of influencing behavior is through 
group pressure”. This serves the organization in two ways. One is that the existing 
attitudes exert a powerful influence on new employees, molding them into the prevailing 
ways of thinking and behaving. Another js the fact that an organization with a positive 


reputation for safety will tend to attract personnel with similar ideals. 
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3; Reflection on Safety Practices 

Reflection upon an organization’s current practice is the final characteristic 
critical to a good safety culture. This 1s best accomplished by mechanisms that provide 
for the generation of accident and incident feedback (Turner, et al, 1989). Two ways of 
achieving this are by implementing a reward system and by ensuring a “no-fault” 
reporting system (Eiff & Mattson, 1998). A reward system offers incentives to 
individuals who report unsafe conditions. A “no-fault” reporting system ensures that no 
retribution is brought to bear against anyone identifying an unsafe situation. A study by 
Westrum (Tumer, 1991) shows that a key element in ensuring reliable performance with 
respect to obtaining feedback on unsafe conditions is an atmosphere in which mistakes 
can be discussed openly and without fear of recrimination. In doing so, these events can 
serve as learning opportunities. All too often, accidents that could have been avoided or 
prevented in the future are covered up, only to be rediscovered when they have produced 


far more serious consequences. 


E. ASSESSING HIGH RELIABILITY ORGANIZATIONS 

Roberts (1988) states that existing organizational research is of little help in 
understanding the organizational processes that defined HROs. Her claim is that the 
current literature claims there was no difference between the way an HRO and other less 
hazardous organizations conduct business. Her belief that HROs embody certain key 
characteristics (such as_ leadership style, management policies, procedures 
standardization, superior training, reward systems) and subsequent research sets the 


ground work that would eventually lead to Libuser’s 1994 Model of Organizational 


Safety Effectiveness (MOSE). This model organizes the traits of HROs into the five 
categories of Process Auditing, Reward System, Quality Assurance, Risk Management 
and Command and Control. It is off this model that Ciavarelli and Figlock (1997) will 
base their CSA in an attempt to assess the HRO aspect of Naval Aviation through the 
perspective of aircrew. 

Baker (1998) will take this process one step further. In attempting to identify the 
contribution that maintenance plays in aviation safety, results from benchmarking and 
mishap data analysis (with respect to Naval Aviation maintenance operations) are utilized 
in conjunction with Civarelli & Figlock’s CSA. Where the CSA assessed safety from an 
aircrew perspective, Baker develops a 67-item questionnaire that assesses safety from the 
perspective of aircraft maintainers. Additionally, Baker’s research constitutes the 
incorporation of a sixth component to the existing MOSE: Communication / Functional 
Relationships. Appendix A contains the elements that comprise each of the six MOSE 
components used in Baker’s research, which concludes with the reduction of his 
questionnaire from 67 to 35 items. 

Continuing where Baker left off, this thesis will further explore the attitude of 
Naval Aviation maintenance personnel with respect to safety. In addition, it will attempt 
to identify any commonality or differences in the safety posture of maintenance personnel 
between different communities within a Naval Reserve Wing. To do so, it will utilize the 
35-item questionnaire; a tool developed out of the concept of these high-risk 


organizations and corporate safety cultures. 
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F. SUMMARY 

High-risk organizations operate in a stressful environment. They must balance 
their day-to-day operations with the possibility of encountering a potentially disastrous 
Situation; a situation that may be set into motion by a seemingly small or even 
unimportant event. But despite this heavy burden, many of these organizations seem to 
prevail by the incorporation of the concept of a safety culture. 

Naval Aviation can benefit from the incorporation of procedures proven 
successful in other HROs. By implementing practices designed to establish and foster an 
organizational safety culture, safety awareness can permeate all levels of Naval Aviation 
operations. From managing the maintenance effort to the maintenance itself, a positive 
safety posture 1s imperative, and there is a need to assess safety attitudes both within and 
across communities to gain an overall perspective. One way to accomplish this 1s 
through the use of the MCAS. 

The organization that wants to promote safety and develop this type of culture 
works with a natural advantage. Turmer (1991) states that people have an innate concern 
with safe, non-harmful ways of working. And due to the self-sustaining nature of these 
safety culture ideals, an organization that implements these practices correctly stands 


only to gain from its investment. 
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If. METHODOLOGY 


A. RESEARCH APPROACH 

The intent of this study is to assess the maintainer’s perception of safety and to 
achieve an understanding of the risks within their organization. This research involves 
analysis of the collected survey results and employs an existing model to identify 
organizational factors in an attempt to improve safety within the aviation maintenance 
community. Additionally, the results from this survey are supplied to the Commander, 
Naval Air Reserve Force (NARF) to keep him apprised of his Type Command’s 
perspective on maintenance safety and aid in developing a proactive safety environment. 
This study involves the analysis of data from a climate assessment survey that is based on 
an existing model of high-reliability organizations. This is done to identify any factors 
that may be utilized in the improvement of safety in aviation maintenance practices. The 
data analysis entails principal component analysis, cluster analysis, analysis of variance, 


multiple comparison analysis and descriptive statistics. 


B. DATA COLLECTION 


iG Subjects 


A total of eight Naval Air Reserve Force (NARF) Squadrons are surveyed. This 
mixture consists of two Helicopter Antisubmarine Light (HSL) Squadrons, two Fixed 
Wing Maritime Patrol (VP) Squadrons and four Fixed Wing Fleet Logistics (VR) 
Squadrons. The HSL Squadrons are based at Naval Air Station (NAS) North Island, CA. 
and NAS JRB, Willow Grove, PA. The VP Squadrons are stationed at NAS JRB, 


Willow Grove, PA and Moffett Field, CA. The Willow Grove data is actually collected 


tO 
in" 


for this thesis and compared / pooled with the Moffett Field data from Baker’s thesis 
research. VR data is obtained from squadrons located at NAS JRB, Willow Grove, PA.: 
NAS New Orleans, LA.; NAS Norfolk, VA; and NAS Brunswick, ME. The survey 
respondents are primarily Enlisted Navy personnel assigned to aviation maintenance 
departments. Their duty status consists of either Selected Reserve (SELRES) or Training 
and Administration of Reserves (TAR) status. No civilians participate in the survey. 


2. Instrument 


The Maintenance Climate Assessment Survey (MCAS) is the final product of 
Baker’s 1998 NPS Master’s Thesis. It consists of a self-administered, group survey with 
15 demographic and 35 maintenance related questions. Appendix B contains Baker’s 
proto-type 67-item questionnaire and Appendix C contains the 35 item MCAS. 
Demographic questions cover the areas of rank, community, service component, shift 
worked, total years of service and total years of aviation maintenance experience. 
Additionally, education level, unit home location, rating, age, and maintenance 
qualifications are addressed in this section. Additionally the demographic questions 
maintain the respondent’s anonymity by neglecting to ask for name, social security 
number or work section. The survey utilizes a Likert type five point rating scale with 
verbal anchors as follows: Strongly Disagree, Disagree, Neutral, Agree, and Strongly 
Agree. 


3. Procedure 


The surveys are conducted in person during either safety stand-downs or during 
the reserve duty drill weekend. An overview is given with respect to the survey and its 
purpose, addressing groups as a whole. The survey is distributed, and any questions that 


arose are promptly answered. Respondents are allowed as much time as necessary to 


26 


complete the surveys with the mean completion time being around 20 minutes. The 
surveys are collected immediately upon completion to allow for maximum 


accountability. 


C. DATA ANALYSIS 


L. Data Tabulation 


Survey demographics and responses are hand entered into an Excel (Microsoft, 
1997) spreadsheet. The spreadsheet consists of rows of respondents and columns of 
survey questions. The first fifteen columns are for demographics, with another thirty-five 
columns representing the actual survey questions. Results are coded in the spreadsheet 
by assigning scores of 1, 2, 3, 4, or 5 corresponding to the Likert scale of Strongly 
Disagree, Disagree, Neutral, Agree and Strongly Agree, respectively. Survey 
questionnaire items that have no response are coded with “Blank” and are dealt with by 
S-PLUS (Mathsoft, 1997) as the data is transformed into a SPLUS 4.5 data frame for 
complete data analysis. 

ja Statistical Analysis 

Basic summary Statistics are developed. Bar charts are used for initial 
familiarization and identification of any possible problems with the data set. Next a 
principal component analysis is performed on the data in an attempt to identify any 
commonality of loadings across the MOSE components. Cluster analysis, utilizing both 
agglomerative nesting and divisive analysis methods are conducted to determine the 
clustering structure of the survey and identify any possible clustering of MOSE 
components. Analysis of Variance (ANOVA) and multiple comparison testing are 


utilized to identify any differences between either the communities surveyed or the 


MOSE components with respect to survey outcome. Descriptive analysis is conducted on 
the data to describe basic and genera] information about the demographic and question 
results. These results include the distribution of survey participants by rank and service 
classifications, and the total sample, mean and standard deviation for each of the 35 


Survey questions. 
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IV. RESULTS 


A. DATA EXPLORATION 

Five hundred three (503) surveys are collected in the process of this thesis. Of 
these, four hundred thirty nine (439) indicate actual maintenance experience. The 
anaylsis of those 439 surveys is addressed here. 


af Bar Chart Analysis 


Bar charts are used to determine its underlying structure. Using the Trellis 
Graphics package in S-PLUS 4.5 (Mathsoft, 1998), bar charts for the responses to each of 
the 35 survey questions are developed. Each community’s results are plotted together on 


a single sheet for comparison (see for example Figure 4). 
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Figure 4. Histogram of Responses for Question 1 by three Communities; VR, VP 
and HSL. 
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The horizontal scale on the bar-plots represent the Likert scale responses (1-5) 
and the vertical scale indicates the number of respondents per community. Overall, the 
data is found to be unimodal in structure and not too lacking in symmetry. Thus it is not 
uncomfortable to use the normal distribution in subsequent analysis. 

je Principal Component Analysis 

Principal Component Analysis is performed on the data in an attempt to identify 
any common loading of MOSE categories across the major contributing components 
accounting for the majority of the variance within the questionnaire. Table 1 shows the 
six MOSE categories used in Baker’s thesis and the questions that comprise each (see 


Appendix A for questions by model component). 


MOSE Category____ 


“Process Auditing Bee 


a 
Reward System 4,13, 14, 25, 32 


2,6 

] 

Quality Assurance 327 We Won 25, 238. 290 
| Risk Management Dal 









Se de ae 4 
| Command & Control [dest ?s 1S, 199835. 35 


Communication & Relationships {| 5, 10, 20, 21, 27, 33 


Table 1. MOSE Categories and Questions. 


Using the S-PLUS statistical system, the command “princomp (X, na.action = 
na.omit, cor = T)” was applied for the principal component analysis. The input “X” is the 
439 by 36 matrix of survey responses. The code “na.action = na.omit” is required in 
order for S-PLUS to handle cases where blanks occur in the data. S-PLUS treats missing 
values (coded NA) by omitting them in the performance of a principal component 
analysis. Forty-six cases are discarded from the original data due to missing values. This 


reduces the number of cases evaluated in the principal component analysis to 393. The 


“cor = T “ imphes that the analysis is based on a correlation matrix rather than a 
covariance matrix. A scree-plot displays each component’s contribution to the total 


variance and is displayed in Figure 5. 
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Figure 5. Principal Component Screeplot. 


Above each bar in Figure 5 is the cumulative percentage of variance accounted 
for by each component and its predecessors. Over one-third of the variance is accounted 
for by the first component, with the remaining variance being spread relatively evenly 
across all other components. This shows evidence that the survey data possess one major 
dimension. 

Figure 6 displays the first five components and the six variables that have 


coefficients or loadings with the largest magnitude for each component. 
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Figure 6. Loadings Plot of the first five Principal Components. 


Identifying the questions that load upon Component | reveals that four of the six MOSE 
categories are represented. Thus, it cannot be said that any single MOSE category 
contributes to the majority of the variance within the survey. This is consistent in 
evaluating the remaining four principal components, which comprise over fifty percent of 
the total variance and represent anywhere from three to five MOSE categories in each 
component. 


3: Clustering 


Cluster analysis is a method of classifying a data set into isolated groups that are 
internally cohesive. Clustering analysis is performed on the 35 questions to determine if 
items clustered according to MOSE category. Two methods by which this can be 
achieved are hierarchical agglomeration and divisive analysis (S-PLUS 4 Guide to 


Statistics, 1997). Though the two methods attempt to achieve the same end product (the 


separation of the data into groups), their algorithms operate in opposite fashions. Thus, it 
is wise to apply both methods to See if their results agree. 

a. Agglomerative Nesting (AGNES). 

Agglomerative nesting is an algorithm that constructs a hierarchy of 
clustering. Considered a “bottom-up” approach, the S-PLUS agglomerative nesting 
algorithm is called “AGNES”. At first, each observation 1s a small cluster by itself. 
During each stage of the algorithm, the two "nearest" clusters are combined to form one 
larger cluster. The algorithm is based on dissimilarities. In AGNES, the distance 
between these clusters is determined via one of two metrics. The currently available 
options are "euclidean" and "manhattan". Euclidean distances are root sum-of-squares of 
differences, and manhattan distances are the sum of absolute differences. If the data set 
is already a dissimilarity matrix, then the distance metric is not required. Clusters are 
merged until only one large cluster remains, which contains all the observations. 

The S-PLUS code for this analysis 1s ““agnes (daisy (t (X)), diss= T).” The 
S-PLUS function “daisy” is used to calculate the dissimilarity matrix for the data set “X.” 
The data set is transposed through the code “t (X)” so as to permute the clustering across 
the survey questions, vice the cases. The “diss=T” statement implies that a dissimilarity 
matrix will be used in the algorithm’s computation of the distance function. The output 
of the AGNES analysis is displayed in a tree type diagram (or dendrogram). The 
command “plot (name)” is used to obtain the dendrogram, where “name” is any arbitrary 
name assigned to the analysis. The resulting AGNES analysis is shown in Figure 7. The 


leaves of the tree are the survey questions. Two branches are joined at the distance of the 
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two clusters being merged, measured by the vertical (height) scale at the left side of the 


dendrogram. 
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Figure 7. AGNES Clustering Dendrogram 


An agglomerative coefficient measures the amount of clustering structure 


of the data set. An agglomerative coefficient of one (1) would indicate a perfect 


clustering structure and a zero (OQ) would indicate no clustering structure. The resulting 


agglomerative coefficient produced from performing AGNES on the data set was 0.42. 


b. Divisive Analysis (DIANA). 


Divisive analysis is a form of cluster analysis that divides a data set into 


groups (clusters) of observations that are similar to each other. Considered a “top-down” 


approach, it begins with one large cluster containing all “n” items. Dissimilarities are 
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used to determine how to divide the data set. The initial split is conducted by finding the 
most disparate ttem in the group (1.e. the one with the highest average dissimilarity to all 
other objects). This item forms the nucleus of what is called the “splinter group” (S- 
PLUS 4 Guide to Statistics, 1997). Then for each item outside the splinter group, average 
distances are computed to all other items and to the splinter group. Items that are closer 
to the splinter group than any other item are added, otherwise it 1s paired with whichever 
item it was closest to. The process continues until the data is split into two groups. In 
each subsequent step of the algorithm, the largest available cluster (i.e. the one with the 
highest dissimilarity) 1s put through the same process as just described with the initial 
split. The process 1s repeated until each cluster contains only a single item. 

The S-PLUS code used to perform the divisive analysis on the data set is “diana 
(daisy (t (X)), diss= T).”” The S-PLUS function “daisy” is used to calculate the 
dissimilarity matrix for the data set “X.” The data set is transposed through the code “‘t 
(X)” so as to permute the clustering across the survey questions, vice the cases. The 
“diss=T” statement implies that a dissimilarity matrix will be used in the algorithm’s 
computation of the distance function. A dendrogram of the results is plotted using the 
code “plot (name), where “name” is any arbitrary name assigned to the divisive 


analysis. Figure 8 shows the output of the divisive analysis run on the data set. 
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Figure 8. DIANA Clustering Dendrogram. 


Evaluation of the AGNES and DIANA dendrograms reveals very similar 
clustering results. Though there is no distinct clustering by MOSE categories, the items 
do group similarly when both agglomerative nesting and divisive analysis were 
performed on the data set. The divisive coefficient, which measures the clustering 
Structure of the data set, was measured at 0.49. This closely reflects the agglomerative 


coefficient of 0.42 obtained through AGNES and further backs the stability of the survey. 


4. Descriptive Statistics 


a Demographics 

Descriptive statistics are developed for the survey respondents. 
Maintenance personnel account for approximately 87.3 percent of the total number of 
Surveys collected for this thesis. A demographic breakdown for each community’s 


maintenance personnel (segregated according to rank) 1s presented in Table 2. The 
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percentage each rank category in Table 2 contributes to the overall total number of 


maintenance personnel both per community and overall is presented in Table 3. 





Table 2. Frequencies of Maintenance Personnel Ranks by Community. 


| 





Table 3. Percentage of Maintenance Personnel Ranks by Community. 


To aid in visualizing the distribution of personnel throughout the 
respective communities, Figure 9 displays the information presented in Table 3. 
Immediate attention is drawn to the disproportionate distribution of personnel among the 
F4-F6 rank category, which makes up approximately 71 % of the total population of 
maintenance personnel. This distribution is also fairly consistent across each community. 
This is something to make note of in the fact that the majority of the maintenance 
performed and direct supervision come from this pool of personnel. This group 
encompasses anywhere from two to twenty-eight years of military service (with an 
average of just over eleven years) and anywhere from one to thirty-four years of 


maintenance experience (with an average of approximately ten and one-half years). 
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Figure 9. Percentage of Maintenance Personnel Ranks by Aircraft Community. 


b. MOSE Component Evaluation 

The results from evaluating the MOSE categories are displayed in Tables 
4 through 10. Ench table contains a list of all questions from the appropriate MOSE 
category, the average response per community for that question on a “1” to “5” Likert 
rating scale, and the overall average response per question. The standard deviation 
corresponding to each question is also included. The analysis reveals that in an 
overwhelmingly majority of the cases, the average response per question 1s very similar. 
This holds in cases where a negative response (i.e. average less than 3.0) was recorded. 
Also noted is that for the two negatively anchored questions in the survey (_.e. 
disagreement is a constructive response), the responses for each reflected accordingly. 
These will be addressed during the individual discussion of each appropriate MOSE 


category. 


38 


The first MOSE category to be evaluated is Process Auditing (PA), which 
is represented by four questions in the survey. All questions were answered positively 


with a mean range from 3.22 to 4.34 (see Table 4). 
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Table 4. Process Auditing MOSE Component Summary. 


The second MOSE category is Reward System (RS), consisting of five 


questions. All questions were answered positively with a mean response ranging from 


3.49 to 4.14 (see Table 5). 





Aircraft Community 
Question [HSE [P| VR | Combinea ‘| 






Table 5. Reward System MOSE Component Summary. 


Table 6 displays the information from the third MOSE category, Quality 
Assurance (QA). Consisting of nine questions, it contains the largest number of 
questions for a single category. All have an average response greater than 3.0. except for 
Question 22, which addresses the area of adequate staffing levels. | 

Of the three communities, two of them answered this question in a 
negative manner with average responses of 2.56 and 2.61 (the HSL and VR communities 
respectively). The VP community answered this question in a positive manner with a 
average response of 3.19. This also corresponds with the VP data from Baker’s (1998) 
which gives this question (though different in number and wording, addressed the issue 
of staffing level) an average response of 3.07. Appendix D contains a mapping of the 
items from Baker’s (1998) prototype questionnaire to the 35 item MCAS for use in cross- 
referencing any items. Though the VP community doesn’t rate this question negatively, 
the fact that the others do should draw attention to it as it addresses the area of sufficient 
staffing which can definitely effect safety in a variety of ways. The remaining questions 


are positively answered with means ranging from 3.16 to 4.35. 
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Table 6. Quality Assurance MOSE Component Summary. 
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Risk Management (RM) is the fourth component in the MOSE model and 
is comprised of five questions. Three of them were answered positively (mean range 
from 3.54 to 4.04. Two of them (Questions 9 and 24) are answered in a negative fashion. 
Question 9 shows all three communities answering negatively with an overall mean 
response of 2.69. This question addresses manning issues and operational commitments. 
This is similar to the negative answering of Question 22 in that it addresses manpower 
issues and should be an area of concern with respect to safety. 

Question 24 refers to the area of personnel turnover and was responded to 
negatively by two of the three communities. The HSL and VR communities both 
responded negatively with mean responses of 2.75 and 2.88 respectively. The VP 
community responded positively with a mean response of 3.21. This coincides with data 
obtained from Baker’s (1998) thesis where the VP community scored a 2.88 to Question 40 
which addresses the same topic but is negatively anchored. With a negatively worded 
question, a score below 3.0 implies a positive response and thus is consistent with the VP 
data obtained during this thesis research. Despite the VP community’s positive response, 
the negative responses obtained from the HSL and VR communities should highlight this 


area for investigation. Table 7 summarizes the results of the Risk Management component. 
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Table 7. Risk Management MOSE Component Summary. 
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The fifth component in the MOSE model is Command & Control (CC). 
Containing six questions, all but one was answered positively with means ranging from 
3.35 to 3.79. Question 23, which addresses the areas of multiple job assignments and 
collateral duties, was scored negatively by all communities (average response of 2.78). A 
negative perception in this area should draw some concem. Table 8 displays the results 


for this MOSE component. 
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Table 8. Command & Control MOSE Component Summary. 


The last component of the MOSE is Communication & Functional 
Relationships (CR) which is comprised of six questions (including the only two 
negatively worded questions in the survey). Of the two negatively worded questions, 
only Question 5, which addresses problems with passdown between shifts was negatively 
responded to. Again, being that the question is negatively worded, a negative response 
(value below 3.0) would indicate a positive perception with respect to safety. The overall 
mean-response of the HSL community to this question was a 3.02, but it was the result of 
averaging together the individual responses of the two squadrons within that community. 
While one rated this question a 2.78 (which indicates a positive response), the other 


averaged a 3.27 (which implies a negative response). Only one squadron answered in 


this fashion and there are two possible explanations: the question was misinterpreted, or 
the squadron did indeed perceive a problem with this area. The resulting average value 
of 3.02, though not heavily negative, points to addressing this issue. 

The other negatively worded question (27), recorded all positive responses 


implying no problems with Maintenance Control’s ability to troubleshoot discrepancies. 


The remaining questions provided a mean range from 3.09 to 3.87 (see Table 9). 
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Table 9. Communication & Functional Relationship MOSE Component Summary. 











(* Indicates A Negatively Worded Question). 


The overall means for all six MOSE categories are presented in Table 10. 
The distributions of these results are depicted via a histogram in Figure 10. Additional 


histograms for each individual MOSE component are enclosed in Appendix E. 







Table 10. Mean Average Response per MOSE Component by Community. 
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Figure 10. Distribution of Model Of Safety Effectiveness Ratings. 


5: Analysis of Variance (ANOVA) 


A two-factor analysis of variance (ANOVA) is run to determine if either 
Community or MOSE category have an affect on the mean survey response. In a two- 
factor ANOVA, there are two hypotheses of interest. The first states that the different 
levels of factor “A” have no effect on the true average response and was denoted Hoa. 
The second states that there was no effect from factor “B” and was denoted Hog. 

The letter “Y’ denotes the number of levels of the first factor of interest (factor 
“A” or Community) and “J” denotes the number of levels of the second factor of interest 
(factor “B” or MOSE category). With J=3 (the three aircraft communities) and J=6 (the 
six MOSE categories), there were JJ (or 18) possible combinations consisting of three 


levels of factor “A” and six of factor ““B.”’ Each such combination was viewed as crossed 


44 


levels of the two factors. There is only one result for each possible cell and the analysis 
utilizes a two-factor ANOVA with one observation per cell. The factors, levels and data 
are shown in Table 11. The ANOVA table obtained from running the two-factor ANOVA 


is given in Table 12. 


MOSE Categories 
Community communi | “FST RM A 


3.618 3.36 3.51 





Table 11. Average Responses per Community in all MOSE Categories. 


aie of | Sum of 
Freedom Squares | Mean Square| F-Value | P-Value 


Community [3 LOmOOTTT | O.0000ESS | OODT6I] O.STIBTOO 
Component | 5 | 0.8124472 | _0.1624894 | 0.68654 | 0.000009 
[Residuals | 10 | 0.0320577 | 0.032058 | | 





Table 12. Analysis of Variance (ANOVA) Table. 


With the p-value of 0.9728, at any sensible level of significance we fail to reject 
Hoa, that the various communities showed no real difference between them with respect 
to their true average pattern of responses over the six survey categories. A boxplot of 
the data visually depicts the similarities between each community’s median values, 
spread, symmetry and outliers (see Figure 11). This depiction corresponds with the high 


p-value obtained in the ANOVA and supports the decision of failing to reject Hoa. 
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HSL VP VR 


Figure 11. Boxplot of Communities. 


The MOSE components, however, proved highly significant with a p-value of 0.0000009. 
This causes us to reject Hog in favor of the claim that different MOSE categories 
correspond to different true average responses. A boxplot of the MOSE components is 


displayed in Figure 12. 





Figure 12. Boxplot of MOSE Components. 
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There is a noticeable difference between the plots of the MOSE components in 
Figure 12 compared to the plots by Communities in Figure 11. Especially noticeable is 
the low average score and low variability of the Communication & Functional 
Relationship (CR) MOSE component. This graph lends support to the p-value obtained 
during the ANOVA and the decision to reject Hog. 

In rejecting Hog, it 1s of interest to attempt to determine which levels of factor “B” 
(or MOSE categories) were different from one another. Tukey’s procedure is a method 
used to determine significant differences between levels of a factor of interest and is 


described next. 


6. Multiple Comparisons in ANOVA 

A multiple comparison test utilizing Tukey’s Procedure (or “T”’ method) is 
conducted to determine which MOSE components proved different from one another. 
Utilizing the Studentized Range probability distribution, simultaneous confidence 
intervals for all pairwise comparisons were computed at a selected alpha (a) level. In 
this case, the pairwise comparing of the means of all six MOSE components resulted in 
fifteen comparisons. The resulting confidence intervals are intervals for the values of all 
pairs of differences between true treatment means. Each interval that doesn’t include the 
value of zero yields the conclusion that the treatment means differ significantly (Devore, 
1995). Figure 13 provides a graphical representation of the treatment mean confidence 


intervals. 
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Simultaneous 95 % Confidence Limits, Tukey method 
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Figure 13. Pairwise Comparisons of MOSE Components Utilizing Tukey’s 


Procedure. 


The overall average for each MOSE component is computed and displayed in 
Table 13. Having the MOSE components arranged in ascending order, the graphical 
display of the confidence intervals (Figure 13) is used to obtain those pairings that 


contained the value of zero. Those components are underlined and shown in Figure 14. 






Table 13. MOSE Components and Associated Means Sorted in Ascending Order. 
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Figure 14. Identifying Statistically Different MOSE Components. 


48 


Any pair of MOSE components not underscored by the same line corresponded to 
a pair of true treatment means that were significantly different. Items within the same 
grouping do not differ significantly. 

Figure 14 shows that the MOSE components can be separated into three basic 
groups. The first group contained the MOSE component of Communication & 
Relationship (CR) by itself. Via Tukey’s procedure, it is judged to be significantly 
different from all other components and receives the lowest marks. The second group 
contained the components of Risk Management (RM) and Command & Control (CC). 
This implies that collectively there was no real difference between the responses received 
for these two categories. Also implied was that the responses received for these two 
categories differed from the remaining four MOSE categories. The third group contained 
Quality Assurance (QA), Process Auditing (PA) and Reward System (RS). These items 
were collectively similar in response, and receiving the highest marks as a group, were 
Significantly different from either Communication & Relationships (CR) or the group 


containing Risk Management (RM) and Command & Control (CC). 
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V. CONCLUSIONS 


A. FINDINGS 

The results of this thesis conclude that the existing MCAS can be used as a tool 
for capturing a command’s attitude with respect to safety. The statistical methods of 
Principal Component Analysis and Cluster Analysis fail to identify any single MOSE 
component that was responsible for controlling the outcome of the survey. Cluster 
Analysis additionally validate the stability of the survey by achieving similar conclusions 
through the use of two diametrically functioning algorithms. 

Through Analysis of Variance (ANOVA) and multiple comparison testing, it is 
determined that there is no discernable difference between the aircraft communities 
surveyed. Though descriptive analysis did discover a few areas of concern that varied 
between communities, the prevailing attitude is one of a positive outlook with respect to 
safety within the aviation maintenance organizations surveyed. Those questions having 
negative responses are individually discussed in Chapter IV and summarized below in 


Table 14. 







[| | Aircraft Community | 
[Question | WSL] VP [VR | Combined | 
[| Ave [Ave | Ave | Ave 



















Table 14. Questions with Negative Responses. 
(* Indicates A Negatively Worded Question). 
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It is interesting to note that of the five questions listed in Table 13, three of them 
deal directly with issues conceming either manpower / staffing levels. But despite the 
varying levels of negativity expressed within these individual questions, the overall 
means for each MOSE category was in fact positive. Analysis of Variance (ANOVA) 
and multiple comparison testing also determined that there is a significant difference 
between the MOSE components with their ability to effect the true average response. It 
is determined that the six categories could be grouped into three distinct subsets and can 
provide a reference for any further refining of this survey. 

During the exploratory phases of this thesis, a few questions are highlighted as 
needing restructuring. This 1s due to the collusion that could result from the addressing 
of certain areas of safety within the same question. This restructuring results in the 
addition of five questions to the MCAS for a total count of 40 items. While trying to 
maintain the number of survey items to a minimum, it is deemed that the clarity obtained 
in the restructuring (and addition of the five questions) was worth the increase. Appendix 


F contains the revised 40-items categorized by MOSE component. 


B. RECOMMENDATIONS 

Intervention strategies should be developed that specifically address each of the 
questions that are identified as potential problem areas. This will target areas of concern 
directly, providing a focused effort. Additionally, this thesis can be used as a starting 
point for numerous future studies. A Marine Corps specific survey could be developed to 
aid Marine Corps Aviation in assessing its safety posture. Similarities between USMC 


and USN operating environments (both procedural and operational) would require 
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minimal modification to the current MCAS to achieve this. If a USMC specific survey is 
developed, results between Navy and Marine Corps squadrons can be compared and 
contrasted. Comparisons can also be made between Regular active duty and Reserve 
units of both services. 

Current mishap data for surveyed squadrons could be obtained from the Naval 
Safety Center. This data could be utilized to compare with patterns developed from the 
survey results between the Squadron’s safety attitude and mishap rate. Finally, other 


analysis methods could be used to further validate this study and its results. 
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APPENDIX A. MODEL OF SAFETY EFFECTIVENESS COMPONENTS. 


COMPONENT 1: PROCESS AUDITING 
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My command adequately reviews and updates safety practices. 

The command has a dedicated program that targets individual training deficiencies. 
My command monitors maintainer qualifications. 

Support equipment licensing 1s monitored in this command. 

Tool control is taken seriously at my command. 

CDIs/QARs routinely monitor maintenance evolutions. 

My command uses safety staff to manage personnel at risk. 

The command uses medical staff to manage occupational hazards and personnel at risk. 


COMPONENT 2: REWARD SYSTEM 
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My command recognizes individual safety achievement through rewards and 
incentives. 

Unprofessional behavior is not tolerated in the maintenance department. 

Supervisors encourage reporting safety concerns without fear of retribution. 
Supervisors discourage violations of SOPs, or NAMP guidelines. 

My MO/MCPO understands if I feel uncomfortable performing maintenance duties 
due to personal issues. 

Violations of SOP, NAMP guidelines, or other procedures are common in my 
command. 

Peer influence discourages violations of SOP, NAMP guidelines or other procedures. 
Personnel are uncomfortable telling supervisors about personal problems including illness. 
Individuals feel free to report safety violations, unsafe performance, or other unsafe 
behavior. 

Our command climate promotes safe maintenance and flight operations. 


COMPONENT 3: QUALITY ASSURANCE 
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My command has established standards and maintains quality control. 
CDIs/QARs are sought after positions in my command. 

Inspectors perform all required actions before sign off. 

To meet operational commitments, supervisors allow “cutting corners.” 
Maintainer staffing is sufficient from shift to shift. 

Proper tools and equipment are available, serviceable and used. 
Required publications are available, current and used. 

Maintenance gripes are either corrected or addressed prior to flight. 

My command has a reputation for quality maintenance. 


. The QA division is respected in my command. 
. Signing off PQS/JQRs/PARs is taken seriously and not gun decked. 
. Maintenance quality on detachments is the same as that in homeport. 
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COMPONENT 4: RISK MANAGEMENT 
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My command temporarily restricts maintainers who are having personal problems. 
Based upon my command’s current manning and assets, it 1s over-committed. 
Supervisors manage hazards associated with maintenance and flight line operations. 
Supervisors are more concerned with mission completion than aircraft maintenance. 
My division CPO is aware of individual daily workload requirements. 

Unsafe conditions are recognized and addressed by M/C, Q/A, or W/C supervisors. 
Personnel turnover negatively affects my command’s ability to operate safely. 

Day and night check have equal workload and are equally stressful/fatiguing. 

I am provided adequate resources (time, personnel, and equipment) to accomplish my 
job. 

Safety decisions are made at the proper command levels. 

Safety is part of maintenance planning, and additional training/support is provided as 
needed. 

Maintainers are never purposely put in an unsafe situation to meet the flight schedule. 


COMPONENT 5: COMMAND AND CONTROL 
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My command ensures all maintainers are responsible and accountable for safe 
maintenance. 

My command ensures the uniform enforcement of SOPs among unit maintenance 
personnel. 

Supervisors communicate command safety goals, programs, and procedures. 
Supervisors are actively involved in the safety program and management of safety 
matters. 

Supervisors set the example for compliance to established maintenance standards. 
Supervisors are responsive to unexpected changes and anticipate potential hazards. 
W/C supervisors are respected by the maintenance chief/officer. 

All maintenance evolutions are properly supervised by qualified personnel. 
Maintenance control is effective in managing all maintenance activities. 

Multiple job assignments and collateral duties adversely affect maintenance. 

In my command, we believe safety is an integral part of all maintenance and flight 
line operations. 

Safety education and training in my command are comprehensive and effective. 

The safety department is respected by supervisors and maintainers. 

Maintenance Safety Petty Officer is a sought after billet in my command. 


COMPONENT 6: COMMUNICATION / FUNCTIONAL RELATIONSHIPS 


IL. 
2 


My command has a problem with passdown between shifts. 

Within my unit, good communication flow exists up and down the chain of 
command. 

Coordination is conducted between the M/C, W/C and QA prior to incorporation of 
TDs. 

Work center supervisors, division CPOs and M/C work well together. 
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Aircraft moves are briefed and detailed personnel are qualified. 

Maintainers are briefed on potential hazards associated with maintenance activities. 
My supervisor shields me from outside pressures, which may affect my work. 
QARs are never pressured by the maintenance supervisors to sign off a gripe. 
Maintenance Control] never troubleshoots aircraft discrepancies. 

10. QARs are viewed as helpful, and QA 1s not “feared” in my command. 

11. I fee] I get all information (internal and external) required to perform my job safely. 
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Purpose: The purpose of this survey ts to try and gain valuable insight into the maintenance community’s 
perception concerning aviation mishaps within the Navy and Marine Corps. Your participation and answers 


APPENDIX B. PROTOTYPE MCAS QUESTIONNAIRE 


will be used as a guide in the Navy’s on-going efforts to lower the aviation mishap rate. 


The first fifteen questions, part I, regard biographical data; information particular to yourself. This 
information will aid in the analysis of your responses. NO attempts will be made to identify individual 


respondents or their organizations. 


Part II has 67 questions pertaining to the maintenance community. Please respond to the questions with 
the answer that most correctly reflects your honest opinion. Using a #2 pencil, completely darken each 


response. 


Thank you in advance for your participation! 


PLEASE RESPOND TO EACH ITEM. 


et 


2. 


POs 


i ie ae 


te 


Ley. 


14. 


toe. 


Your rank? ( ) E-1 - E-3 ( )E-4 - E-6 ( )CPO E-7 + ( )Officer 


Your comminity ? 


VFA ( ) HS oo VMFA ( ) VF ( ) HSL ( ) 
VMA ( ) He ae VP (i 2 HCS ( ) Ve.) 
VR oe, VQ ( ) VAQ ( ) VAW (3 
Weur Gesiqneror? 4 bDO, 152%, etc )? / NEC i 
Are you currently a department head? ( ) Yes (Tie NS 
Your service? ( ) USN ( ) USNR TAR ( ) SELRES ( ) Other 
moma Soiree”. { “)> Dx ( ) NX ( ) Midax { ) Other, specify 


Total years of service? 
Total years of Aviation Maintenance experience? 


A-School graduate? ( ) Yes (-), No ( ) N/A 


Education level:( ) GED ( ) High School ( )College, # of years 


Unit home location? ( )East coast ( )West Coast ( )Other 
Your rating? ( )AD/AM ( ) AE/AT ( )PR/AME ( )AO ( )Other 
Your age? ( ) 17-20 Ce 2i-2s ( ) 25-30 ( ) 30+ 
Current maintenance qualifications? 
( )Safe for Flight ( )QAR 
( ) ere ( )Supervisor 
( )SPO ( )N /A 


Daey : ( ) Shore ea) Sea 


a7 


Part 


1. Mycommand adequately reviews and updates 
safety practices. 


i) 


The command has a dedicated program that 
targets individual training deficiencies. 


3. My command monitors maintainer qualifications. 


4. Support equipment licensing is monitored 
in this command. 


5. Tool Control is taken seriously at my command. 


6. My command recognizes individual safety 
achievement through rewards and incentives. 


7. Unprofessional behavior is not tolerated in the 
maintenance department. 


8. My command has a problem with passdown 
between shifts. 


9. My command follows established standards 
and maintains quality control. 


10. CDIs/QARs are sought after positions in my 
command. 


11. My command temporarily restricts maintainers 
who are having personal problems. 


12. Based upon my command’s current manning 
and assets, it 1s over-committed. 


13. My command ensures all maintainers are responsible 
and accountable for safe maintenance. 


14. My command ensures the uniform enforcement 
of SOPs among unit maintenance personnel. 


15. Within my unit, good communication flow 
exists up and down the chain of command. 


16. Coordination is conducted between M/C , W/C 
and QA prior to incorporation of TDs. 


17. Inspectors perform all required actions before 
sign off. 


18. Supervisors encourage reporting safety concerns 
without fear of retribution. 


19. Supervisors discourage violations of SOPs, 
or NAMP guidelines. 
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24. 


py 


26: 


213 


28. 


Pig 


30. 


31. 


32. 


a2. 


34. 


Sk 


36. 


a7. 


38. 


To meet operational commitments, supervisors 
allow “cutting corners”. 


Supervisors manage the hazards associated with 
maintenance and flight line operations. 


Supervisors are more concerned with mission 
completion than aircraft maintenance. 


My division CPO is aware of individual 
daily workload requirements. 


Unsafe conditions are recognized and 
addressed by M/C, Q/A, or W/C supervisors. 


Supervisors communicate command safety goals, 
programs and procedures. 


Supervisors are actively involved in the safety 
program and management of safety matters. 


Supervisors set the example for compliance to 
established maintenance standards. 


Supervisors are responsive to unexpected changes 
and anticipate potential hazards. 


W/C supervisors are respected by the maintenance 
chief/officer. 


All maintenance evolutions are properly supervised 
by qualified personnel. 


Work center supervisors , division CPOs and 
M/C work well together. 


Aircraft moves are briefed and detailed personnel 
are qualified. 


Maintainers are briefed on potential hazards 
associated with maintenance activities. 


My supervisor shields me from outside 
pressures which may affect my work. 


QARs are never pressured by the maintenance 
supervisors to sign off a gripe. 


My MO/MCPO understand if I fee! uncomfortable 
performing maintenance due to personal issues. 


Maintainer staffing is sufficient from shift to shift. 


Maintenance control is effective in managing 
all maintenance activities. 
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45. 


46. 


47. 


48. 


49. 


50. 


a1. 


a 


>. 


54. 


>): 


56. 


Bye 


Multiple job assignments and collateral duties 
adversely affect maintenance. 


Personnel turnover negatively affects my 
command's ability to operate safely. 


Violations of SOP, NAMP guidelines, or other 
procedures are common in my command. 


Proper tools and equipment are available, 
serviceable and used. 


Maintenance Control never troubleshoots 
aircraft discrepancies. 


Required publications are available, current, 
and used. 


Maintenance gripes are either corrected or 
addressed prior to flight. 


CDIs/QARs routinely monitor maintenance 
evolutions. 


My command has a reputation for quality maintenance. 


The QA division is respected in my command. 


Signing off PQS/JQRs/PARs is taken seriously 
and not gun decked. 


Maintenance quality on detachments is the same 
as that in homeport. 


Day and Night Check have equal workload and 
are equally stressful/fatiguing. 


QARs are viewed as helpful, and QA is 
not “feared” in my command. 


Peer influence discourages violations of SOP, 
NAMP guidelines or other procedures. 


Personnel are uncomfortable telling supervisors 
about personal problems including illness. 


I am provided adequate resources (time, personnel 
and equipment) to accomplish my job. 


In my command, we believe safety is an integral 
part of all maintenance and flight line operations. 


I feel I get all information (internal and external) 
required to perform my job safely. 
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65. 


66. 


67. 


Individuals feel free to report safety violations, 
unsafe performance, or other unsafe behavior. 


My command uses safety staff to manage 
personnel at risk. 


Our command climate promotes safe maintenance 
and flight operations. 


Safety decisions are made at the proper 
command levels. 


Safety is part of maintenance planning, and 
additional training/support is provided as needed. 


Maintainers are never purposely put in an 
unsafe situation to meet the flight schedule. 


Safety education and training in my command are 
comprehensive and effective. 


The safety department is respected by 
supervisors and maintainers. 


Maintenance Safety Petty Officer is a sought after 
billet in my command. 


The command uses medical staff to manage 
occupational hazards and personnel at risk. 
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APPENDIX C. 35-ITEM MCAS QUESTIONNAIRE 


Purpose: The purpose of this survey is to try and gain valuable insight into the maintenance community’s 
perception concerning aviation mishaps within the Navy and Marine Corps. Your participation and answers 
will be used as a guide in the Navy’s on-going efforts to lower the aviation mishap rate. 

The first fifteen questions, part I, regard biographical data; information particular to yourself. This 
information will aid in the analysis of your responses. NO attempts will be made to identify individual 
respondents or their organizations. 

Part II has 35 questions pertaining to the maintenance community. Please respond to the questions with 
the answer that most correctly reflects your honest opinion. Using a #2 pencil, completely darken each 
response. 

Thank you in advance for your participation! 


PLEASE RESPOND TO EACH ITEM. 


a Your rank? ( ) E-1 - E-3 ( )E-4 - E-6 ( )CPO E-7 + ( )Officer 

veo OUr COMmmuUnILY - 
VFA ( ) lols) (=) VMFA ( ) VF ( ) Hsia ) 
VMA ( ) 13(@ (-) VE ( ) HCS ‘e VX (=?) 
VR ( ) VO ( ) VAG eo VAW 3) 

Bee Our designator? { LDO, I52x%, ete )? J ONES 

4. Are you currently a department head? () Yes ( ) No 

5. Your service? ( )}) USN ( ) USNR TAR ( ) SELRES ( ) Other 


Semeervour Shirt? ( ) Dx ( ) NX ( ) Mids ( ) Other 

7. Total years of service? 

8. Total years of Aviation Maintenance experience? 

9. A-School graduate? ( ) Yes ( ) No ( ) N/A 
HOmmeaucation level:( ) GED ( ) High School ( jecollege, tyrs___ 
11. Unit home location? ( )East coast ( )West Coast ( )Other 
ieeecour rating? ( )AD/AM ( ) AE/AT ( )PR/AME ( )AO ( )Other 
if, YOur age? (. } ei7T=20 ie ae be as ( ) 25-30 ( ) 30+ 


14. Current maintenance qualifications? 


( )Safe for Flight ( )QAR 
( )eebi ( )Supervisor 
( ) SPO ( JN /A 


toe Duty : ( ) Shore i) Sea 
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Part Hl 


l. 


to 


10. 


le 


My command has a dedicated program that 

targets individual training deficiencies and ensures 

the uniform enforcement of SOPs among maintenance 
personnel. 


My command monitors maintainer qualifications 
and support equipment licensing. 


My command has a reputation for quality 
maintenance and tool control is taken seriously. 


Unprofessional behavior is not tolerated in the 
maintenance department. 


My command has a problem with passdown 
between shifts. 


My command adequately reviews and updates 

safety practices, follows established standards and 
maintains quality control, ensuring that all maintainers 
are responsible and accountable for safe maintenance. 


QARs/ CDIs and Maintenance Safety Petty Officer are 
sought after billets in my command. 


. Medical and safety staff are used to help identify, 


manage, and temporarily restrict personnel with personal 
issues and those who pose a risk to safe maintenance in 
this command. 


Based upon my command’s current manning 
and assets, 1t 1s not over-committed. 


Within my unit, good communication flow 
exists up and down the chain of command. 


Maintenance control is effective in managing all 
maintenance activities, coordinating between M/C, 
W/C, and QA prior to the incorporations of TDs. 


Inspectors perform all required actions before 
sign off. 


. Safety concerns or unsafe hazards associated 


with maintenance/flight line operations can be 

reported without fear of retribution knowing that the 
WIC, Q/A, or M/C supervisors will address and manage 
them for proper corrections. 


Violations of SOP, NAMP guidelines or other 
procedures are discouraged in this command. 
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26. 


ae 


28. 


29 


30. 


Supervisors are more concerned with proper aircraft 
maintenance than mission completion and do not 


allow cutting corners to meet operational commitments. 


My supervisors are aware of individual 
daily workload requirements and recognize safety 
achievements through rewards and incentives. 


Supervisors communicate command safety goals, 
programs and procedures. 


WI/C supervisors are respected by the maintenance 
chief/officer. 


Qualified personnel properly supervise all 
maintenance evolutions and maintainers are 
briefed on the potenual hazards associated with 
maintenance activities. 


My supervisor shields me from outside 
pressures which may affect my work. 


QARs are never pressured by the maintenance 
Supervisors to sign off a gripe. 


Maintainer staffing is sufficient, is equally worked and 
is equally stressed / fatigued from shift to shift. 


Multiple job assignments and collateral duties 
do not adversely affect maintenance. 


Personnel turnover does not affect my 
command’s ability to operate safely. 


Violations of SOP, NAMP guidelines, or other 
procedures are not common in my command. 


Proper tools and equipment are available, 
serviceable and used and I am provided adequate 
resources (time, personnel) to accomplish my job. 


Maintenance Control never troubleshoots 
aircraft discrepancies. 


Required publications are available, current, 
and used. 


. The QA division is respected and CDIs / QARs 


routinely monitor maintenance evolutions ensuring 
that maintenance gripes are either corrected or 
addressed prior to flight. 


Signing off PQS/JQRs/PARs is taken seriously, 


not gun decked and maintenance quality is as high 
on detachments as it is in homeport. 
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31. Safety is an integral part of this command’s 


maintenance planning/flight line operations, where 
QARs are helpful and the QA division is not “feared” 


Personnel are comfortable telling supervisors 
about personal problems including illness. 
33. I feel I get all information (internal and external) 
required to perform my job safely, and feel free to 


report safety violations, unsafe performance or other 
unsafe behavior. 


34. Maintainers are never purposely put in an 


unsafe situation to meet the flight schedule. 
35 Safety education and training in my command are 


comprehensive and effective and the safety department 
is respected by the supervisors and maintainers. 
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APPENDIX D. MOSE MAPPING INDEX 


Purpose: The purpose of this Appendix is to display a cross-reference of the questions 
from the 67-item Proto-type survey that composed the 35-item Final MCAS questions. 


Final MCAS Proto-Type 
Question # MCAS Question 
# (s) 
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APPENDIX E. MOSE COMPONENT DISTRIBUTIONS. 
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Figure 15. Process Auditing MOSE by Community. 
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Likert Scale 1: Strongly Disagree...5: Strongly Agree 


Figure 16. Reward System MOSE by Community. 
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Figure 17. Quality Assurance MOSE by Community. 
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Figure 18. Risk Management MOSE by Community. 
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Figure 19. Command & Control MOSE by Community. 
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Figure 20. Communication & Functional Relationships MOSE by Community. 
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APPENDIX F. 40-ITEM MCAS QUESTIONNAIRE. 
The items for the 40 Question MCAS are listed according to MOSE Component. 


COMPONENT 1: PROCESS AUDITING 


———s 
ry 


ese 


The command adequately reviews and updates safety practices. 

The command monitors maintainer qualifications and has a program that targets 
training deficiencies. 

The command uses safety and medical staff to identify/manage personnel at risk. 
CDIs/QARs routinely monitor maintenance evolutions. 

Tool Control is taken seriously in the command and support equipment licensing is 
closely monitored. 

Signing of PQS/JQRs/PARs is taken seriously and not gundecked. 


COMPONENT 2: REWARD SYSTEM and SAFETY CLIMATE 


Il; 


Our command climate promotes safe maintenance and flight operations. 
Supervisors discourage SOP, NAMP guideline, or other procedure violations and 
encourage reporting safety concerms without fear of retribution. 

Peer influence discourages SOP, NAMP guideline, or other procedure violations and 
individuals feel free to report safety violations, unsafe performance, or unsafe 
behaviors. 

Violations of SOP, NAMP guidelines, or other procedures are not common and 
unprofessional behavior is not tolerated in the command. 

The command recognizes individual safety achievement through rewards and 
incentives. 

Personnel are comfortable approaching supervisors about personal issues/illness. 
Maintenance Safety Petty Officer, Quality Assurance Representative, and Collateral 
Duty Inspector are sought after billets in the command. 


COMPONENT 3: QUALITY ASSURANCE 


Il. 


o 


See 


The command has a reputation for quality maintenance and has set standards to 
maintain quality control. 

The QA Division and Safety Department are respected in the command and are seen 
as essential to mission accomplishment. 

QARs/CDIs perform all required actions before sign-off and are never pressured by 
maintenance supervisors. 

Maintenance quality on detachments is the same as that at home station. 

Required publications/tools/equipment are available, current/serviceable, and used. 
QARs are viewed as helpful, and QA is not “feared” in my command. 
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COMPONENT 4: RISK MANAGEMENT 


Multiple job assignments and collateral duties adversely affect maintenance. 

Safety is part of maintenance planning, and additional training/support is provided as 
needed. 

Supervisors recognize unsafe conditions and manage the hazards associated with 
maintenance and flight line operations. 

I am provided adequate resources (time, personnel and equipment) to accomplish my 
job. 

Personnel turnover does not negatively affects the commands ability to operate safely 
and based upon my its current manning/assets, it is not over-committed. 

Supervisors are more concerned with aircraft maintenance than mission completion, 
and do not permit cutting corners or purposely putting maintainers in unsafe 
situations to meet the flight schedule. 

Maintainer staffing is sufficient from shift to shift, and Day/Ni cht Check have equal 
an equally stressful/fatiguing workload. 

Supervisors shield personnel from outside pressures that may affect their work, and 
are aware of individual workload and personal issues. 


COMPONENT 5: COMMAND AND CONTROL 


Ih. 


n 


8. 


The command temporarily restricts maintainers who are having personal problems. 
Safety decisions are made at the proper command levels, and CC W/C supervisors 
are respected by the maintenance chief/officer. 

Supervisors communicate command safety goals, programs, and procedures, and are 
actively involved in the safety program and management of safety matters. 
Supervisors set the example for compliance to maintenance standards and ensure 
uniform enforcement of SOPs, NAMP guidelines, and other procedures among 
maintenance personnel. 

In my command, safety is an integral part of all maintenance and flight line 
operations and all maintainers are responsible and accountable for safe maintenance. 
Safety education and training in my command are comprehensive and effective. 

All maintenance evolutions are properly briefed, supervised, and staffed by qualified 
personnel, including flight line activities such as aircraft moves. 

Maintenance control is effective in managing all maintenance activities. 


COMPONENT 6: COMMUNICATION / FUNCTIONAL RELATIONSHIP 


le 


Good communication flow exists up and down the chain of command and I get all the 
information required to perform my Job safely. 

Work center supervisors, division CPOs, QA, and M/C coordinate their actions, 
including the incorporation of TDs. 

My command has effective pass-down between shifts. 

Maintenance Control always troubleshoots aircraft discrepancies and gripes are either 
corrected or addressed prior to flight. 

Maintainers are briefed on potential hazards associated with maintenance activities. 
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